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ates to e 

to an accessing of the image sequence with the aid of the search features. 



The^invention relates to a method for storing search features of an image sequence and 
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Today, audiovisual information (image sequences) of all kinds is stored in digitized 
form at various locations. In order to use this audiovisual information, it is necessary 
to find the relevant data first. 

The MPEG standards of image compression have long been known to the person 
skilled in the art. 

First, to define specific terms used herein: An image sequence designates a series of 
15 ^ related images. An i mago Dccno is a real subset of an image sequence and is initiated 
by an kitraimagg that is defined in MPEG or ITU, respeetiveiy . 

The- obtecl of t he invention ' is to set fo A a method for storing search features of an 
image sequence/image scene and to make it possible to access this image 
2 0 sequence/image scene via these search features. 



-Thfe- objcct is achieved according to the fcatur c s -o£ patent cl a im L 

^ The.method makes it possible to store search features of an image sequence and to 
K 

2 5 access the image sequence with the aid of the search features, said search features 
being determined from the image sequence. The search features are stored together 
with the image sequence. The image sequence is accessed in the event that a 
predetermined item of information is contained in the search features. 



A develo p ment of the invention consi a tG in the search features-fe^iag audio data and/or 
video data of the image sequence. 

Another d e velopment consists in th e ability to - a ccess- an image within the image 
5 sequence w^ith the aid of the search features, which in turn contain a reference (a 
pointer) on their part. 

The access can be accomplished in that the reference is offset in time within the image 
sequence and thereby references the image. Access to the image is thus guaranteed. 
1 0 The reference addresses the image and with this a beginning within the image 
sequence. 

^ Jn- a further-d ev elopme nt, it is possible to prefix search features to the image sequence, 
to append them to one end of the image sequence, or to store them within the image 

15 ^ sequence. This is preferably accomplished with the aid of an intraimagQ that is 
defined in MPEG or ITU, which initiates an image scene and to which the search 
A features can be prefixed. Several image scenes with a respective tntraimage can be 
contained within the image sequence. 

2 0 Aii ol hci dev e l op m en t c onsists in s tori ng each individual image scena in a database . 
A - and - thus guaranteeing access to the image scene. 

An addit i onal de vetopmenrconsi sts hi stuiing separate search feature^in the image 
sequence for individual objects that are defined according to the MPEG or ITU 
2 5^ standard, Fcspectivcly '. It is possible in this way to offer entry points that are related to 
such objects and linked to the search features. 



Another development consists in unambiguously identifying the search features using 
-^ a prcdctcrmincd -s tail code (heade r^ — » 
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Since the search features are stored together with the image sequence, a 
comprehensive image database can be compiled from individual image sequences 
such as this. On the other hand, a global, preferably external, reference list containing 
the search features and corresponding references to the image sequence complicates a 
5 modular construction of the image sequence (above all when this consists of several 
image sequences that are to be exported to other databases individually). 

Further developments of the invention also derive from the dependent claims. 

1 0 -p^emplifying embodiments of the mvemion are detailed with the aM'of'the Figuiesr 

^ Sliown a re: 

Ftgnre-i a-bto ck diagram illtt st rating a me t ho d -fer-ster-ing-seai cl i f eatuies u[~mi — • 

4 mag e s equence and illustrating an a eees sing of the im age-sequencgj^ith 
15 -t he aid of the search f e atur es, - 

Figurer2 Various possibilities for the common storing of search teatures wrEB~The* 
HiTlage sequence, 

Figure-3 a s ke t c h il lus tr at i ng the sea rch features and appertaining reterencestfaat 

deFe fcicncc [sic] the image data : 

2 0" 

In step la of Figure 1, search features are determined from the image sequence. To 
this end, images are sought that correspond to a given pattern image completely or in 
parts. This is achieved in that the pattern image is first analyzed with respect to 
predetermined features, particularly the predominant color or typical shapes that 
2 5 occtir. These features are stored in a database and referenced with a pointer indicating 
the image material that is to be referenced; that is, the found pattern image within the 
image sequence. The search features are stored together with the image sequence in a 
step lb. Finally, it becomes possible to access the image sequence with the aid of the 
search features in a step Ic in that a predeterminable feature is compared to the search 



^ features, and, if agreement was found [sic] , the procedure skips to the image sequence 

A 

within the image sequence referenced by the search features. 

It should be noted that it is not only possible to reference video data, exclusively, but 
5 that pointers indicating audio data can also be stored just as well. 

In Figure 2 various possibilities for storing search features together with the image 
sequence are illustrated. 

10 Figure 2a depicts an image sequence GOP (Group of Pictures, as defined in MPEG 
standards), containing a header H and image information BE. A time axis t running 
vertically from the top down applies to all Figures 2a to 2b. This indicates that the 
time characteristic is a time characteristic of an image series that is stored as an image 
sequence. 

15 

In Figure 2a search features M are prefixed to the image sequence GOP. The search 
features M are preferably stored in a form that can be read easily as a list in plaintext 
(ASCII code). Long lists of search features can also be stored compressed. A pointer 
PTR indicating an image in the image sequence is connected to each individual search 
2 0 feature, whereby this image indicates the beginning of an image series within the 
image sequence, which series is linked to the search feature. 

In Figures 2a to 2d such references (pointers) are represented as arrows PTR. 

2 5 In Figure 2b, the search features M are affixed behind the image sequence GOP. 
Otherwise the same applies here as in Figure 2a. 

As illustrated in Figure 2c, the search features M can be stored within an image 
sequence GOP. Accordingly, references PTR that are linked to the search features M 
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must refer to image information before the search features M (above the search 
features M in Figure 2c) or to image information after the search features M (below 
the search features in Figure 2c) in terms of time. 

5 Finally, Figure 2d depicts the coding of individual objects (audio objects and/or video 
f\ objects) - that is po s siblc ^according to the MPEG or ITU standards^ ^0Gpcctij;cly . For 

this purpose, separate search feature sets M31 and M32 to each object 01 and-02-are 
A sup pleme nted [sic] An audiovisual scene can e ontai ft several sets of search features 

M1,M2, M31 and M32. 

10 

In Figure 2d the search feature sets Ml and M2 relate to image data BD generally, 
^ whereas the search feature sets M3 1 and M32 respectively contain search features, 

whose entries (references) within the image series which are divided according to 
/3 objects 01 and 02^refer to corresponding locations that are to be referenced. 
15 

Figure 3 depicts an organization of a list of search features^ and their relation to the 
image data BD. The vertical course of a time axis t from the top down again applies. 
The search features M in the form of a list are prefixed to the image sequence GOP 
containing the header H and the image data BD. The list contains several search 
2 0 features SMI and SM2. Each search feature respectively comprises a descriptive 

component INFOl and INF02 and a respective reference (pointer) PTRl and PTR2 
indicating an image within the image data BD. 

As described above, for an image sequence that is compressed according to the MPEG 
2 5 standard, the list can be logically subdivided so that specific search features S^j result 
with respect to predeterminable objects within the image sequence that is compressed 
^ according to the MPEG standard.-fste} To differentiate two objects, every second 

search feature SMjj is allocated to a first object, and the other search feature SM(2i+i) is 
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allocated to a second object (i=0,l,2,3,- )- Accordingly, different search feature sets 
can be created for several objects within the same image sequence. 



